Interim Report on Establishing MIR/MDL Evaluation Frameworks: Commentary on Consensus Building

“ The MIR / MDL Evaluation Project White Paper Collection ” Edition # 3 Establishing Music

The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future

Computer Music Journal, 28:2, pp. 12–23, Summer 2004 2004 Massachusetts Institute of Technology. Music Information Retrieval (MIR) is a multidisciplinary research endeavor that strives to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world’s vast store of music accessible to all. Some teams are developin...

Toward the scientific evaluation of music information retrieval systems

This paper outlines the findings-to-date of a project to assist in the efforts being made to establish a TREC-like evaluation paradigm within the Music Information Retrieval (MIR) research community. The findings and recommendations are based upon expert opinion garnered from members of the Information Retrieval (IR), Music Digital Library (MDL) and MIR communities with regard to the constructi...

The International Music Information Retrieval Systems Evaluation Laboratory: Governance, Access and Security

The IMIRSEL (International Music Information Retrieval Systems Evaluation Laboratory) project provides an unprecedented platform for evaluating Music Information Retrieval (MIR) and Music Digital Library (MDL) techniques, by bringing together large corpora and significant computational resources with the necessary rights management and technical infrastructure to support a variety of MIR/MDL re...

Whither Music IR Evaluation Infrastructure: Lessons to be Learned from TREC

Benchmark tasks are a powerful method for advancing the state of the art in a field. The music information retrieval community recently acknowledged the utility of such tasks by resolving to create an evaluation framework for music information retrieval (MIR) tasks and music digital libraries (MDL). This paper describes the processes used in the Text REtrieval Conference (TREC) evaluations to c...